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ABSTRACT 
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using teacher, student, and parent perception data on school climate scales. 
The Success for All program is intended to ensure that every child learns to 
read in the early grades and groups students according to reading level for 
one 90-minute reading period per day. The model also incorporates a Family 
Support Team to encourage parental involvement in the school. School level 
comparison showed the model had a positive effect on achievement test scores 
and attendance. At the student level, results showed a reading effect, 
especially in basic reading skills. Teachers, students and parents at program 
schools showed more positive perceptions of school climate, educational 
quality, and job satisfaction than comparison groups. Implications for the 
accountability and reform movements are discussed. Teacher, student, and 
parent surveys are appended. (Contains 34 references.) (DB) 
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Abstract 

Schools across the nation are implementing models that will enable all children to 
meet state-established academic standards. One of the nation's comprehensive 
school reform models is Success for All. This study assessed the impact of the 
program on elementary school students (n = 217) in standardized reading scores 
as well as attendance and disciplinary measures when compared to control 
students (n = 132). Program (n = 3) and control schools (n = 3) were also 
compared using teacher, student, and parent perception data on school climate 
scales. Significant effects were not seen on every measure, but the consistent 
direction and magnitude of the effects showed benefits for SFA students when 
compared to control students. Implications for educational policy and practice are 
discussed. 

Keywords: Elementary School Students; High Risk Students; Academic 
Achievement; Early Intervention; Reading Achievement; School Restructuring; 
Urban Schools. 
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Educating Students Placed At Risk: Evaluating the Impact of Success for All in 

Urban Settings 

Over the last two decades, numerous national studies and reports have 
documented both the struggles and failings of public education. Educators, policy- 
makers, and researchers alike concluded that a large number of schools, 
particularly in high poverty urban centers, were ineffective at meeting the needs 
of diverse student populations (Cooper, 1998). In an effort to assist schools in 
making curriculum changes, aid in instructional delivery, and strengthen the 
organizational structure of the schools, an abundance of school-wide reform 
models have emerged. If educators have learned anything about school reform, it 
is that a piecemealed approach to changing poor classroom practice is a losing 
battle. A collection of isolated programs does not add up to school-wide 
improvement (Northwest Regional Educational Laboratory, 2001). 

The comprehensive approach to school-wide improvement is not new. A 
line of inquiry referred to as effective school research were forerunners in taking a 
comprehensive approach to school improvement. These studies attempted to 
identify the characteristics of schools that make them instructionally effective for 
disadvantage students (Brookover, Beady, Flook, Schweitzer, & Wisenbaker, 

1979; Clark, Lotto, & McCarthy, 1980; Edmonds, 1979; Weber, 1971). 

One of the nation's comprehensive school reform models is Success For 
All (SFA). SFA is designed for elementary schools (PreK-6) to ensure that every 
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child learns to read in the early grade levels (Slavin, Madden, Dolan, Wasik, 

Ross, Smith, & Dianda, 1996). The primary goal of SFA is to prevent reading 
problems in the first place and to intervene swiftly and intensively if problems do 
appear (Northwest Regional Educational Laboratory, 2001). According to 
Madden and Slavin (1999), the first goal of reform should be to ensure that every 
child, regardless of home background, home language, or learning style, achieves 
the success that he or she so confidently expected in kindergarten. 

In the SFA program organizational structure, students are grouped 
according to reading-level for one 90-minute reading period per day while the rest 
of the day they are assigned to regular age-grouped grades. At eight-week 
intervals, teachers evaluate the performance reading level of students to (a) 
determine who requires reading tutors, (b) reassign reading groups, (c) suggest 
other adaptations in students' programs, and (d) identify students who need other 
types of assistance (e.g., screening for vision and hearing problems). The model 
also incorporates a Family Support Team (FST), to encourage parental 
contribution and involvement in the school. The FST is designed to help increase 
parental involvement. SFA schools have a Program Facilitator who is responsible 
for assisting teachers on the implementation of the curriculum and classroom 
management. Teachers receive three days of in-service training at the beginning 
of the school year. Throughout the year the program facilitator and technical 
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support staff provide professional development on topics such as classroom 
management, instructional pace, and cooperative learning. 

Established in 1987 by Robert Slavin and colleagues from Johns Hopkins 
University, SEA has been implemented in about 15,000 schools in 48 states 
throughout the nation. It is considered one of the most successful and extensively 
researched whole-school change models (Borman & Hewes, 2001; Cooper, 1998; 
Cooper, Madden, & Slavin, 1997; Madden & Slavin, 1999). Developers and third- 
party research work has been done in Memphis, Tennessee (Ross, Sanders, & 
Wright, 1998), Fort Wayne, Indiana (Ross, Smith, & Casey, 1997), Flint, 
Michigan (Madden & Slavin, 1999), and Baltimore, Maryland (Slavin, Madden, 
Dolan, Wasik, Ross, & Smith, 1994). 

Madden, Slavin, Karweit, Dolan and Wasik (1993) investigated the 
longitudinal effects of SFA on inner-city elementary schools in Baltimore. The 
purpose was to identify outcomes of student reading achievement and other 
outcomes in elementary schools serving large number of disadvantaged students. 

A total of five SFA schools were studied over a three-year period with 
comparisons with matched students in matched schools. Control schools were 
matched on percentage of students receiving free lunch, historical achievement 
level, and other factors. Within each matched school, students were individually 
matched on standardized achievement test scores from the spring before SFA 
implementation. The reading test data were analyzed using multivariate analyses 
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of variance (MANCOVA), with pretest data as covariates and scores on the 
reading scales as dependent measures. Univariate analyses of covariance 
(ANCOVA) were also computed for each dependent measure. Additionally, 
separate analysis was conducted for students who scored in the lowest 25% of 
their cohorts on the pretests. The results found strong positive effects on 
administered reading measures in most schools for students who have been in the 
program since the first grade. 

Ross et al (1994) conducted an investigation to assess the impact of SFA 
on students at risk of failing to learn to read. The student examined the effects of 
SFA program in four cities: (a) Memphis, Tennessee; (b) Montgomery, Alabama; 
(c) Fort Wayne, Indiana; (d) Caldwell, Idaho. Specifically the study sought to 
assess student achievement when the SFA program was introduced in setting 
geographically removed from program developers. A comparison of individual 
reading test results with those of matched control groups indicated advantages at 
three of the sites. Of particular interest was the assessment of the lowest achieving 
sample. Advantages were detected for the lowest achieving 25 percent of students 
relative to their control counterparts at all four sites. Results also indicated that the 
SFA program could be replicated at distant locations and with limited monitoring 
by program developers. The systematic procedures of the program facilitated the 
replication process. 
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Ross and colleagues (1997) examined the impact of SFA in school 
performance across three grades on both individually and state-mandated 
standardized reading tests. The study used program and matched-control students. 
Findings indicated more positive effects for SFA students, some decline in effects 
over time, and greater effects for minority than for non-minority students. The 
same year, Jones, Gottfredson, and Gottfredson conducted another similar 
evaluation to assess the impact of the SFA program in early grades elementary 
schools. The researchers also used a control group methodology. The SFA 
program was implemented for three years and under difficult circumstances. The 
findings of the comparison of the SFA program and control group were mixed 
results. Findings for 1 13 SFA students and 59 comparison students showed 
positive effects for the kindergarten component but largely negative effects in 
grades two and three. The researchers recommended that future evaluations 
should assess the impact of the program when SFA students reach the third grade 
level. 

More recently, Borman and Hewes (2001) conducted an investigation on 
the long-term outcomes of SFA. Sustained effects on student achievement was 
proposed and investigated through treatment (n = 1310) and control (n = 1730) 
student secondary data of the Baltimore City Public School System. The treatment 
students included all students from the original SFA elementary schools and the 
control students were matched on demographic characteristics and the amount of 
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funding their school received. Three dependent variables were of primary interest: 
(a) achievement, (b) grade-level progression (or retention), and (c) special 
education placements. The investigators utilized an ANCOVA, controlling for 
pretest score. Findings indicated that there were statistically significant 
differences between SFA and control students. When controlling for kindergarten 
pretest differences, SFA students had higher eighth grade CTBS/4 reading and 
math scale scores, in comparison to control students. The findings showed 
statistically significant differences between SFA and control students with SFA 
students spending fewer years enrolled in special education and a lower number 
of retentions. 

To date, little SFA research has been conducted in Kentucky, a state 
characterized for its innovative educational reform environment. In 1990, the 
Kentucky General Assembly passed the Kentucky Educational Reform Act 
(KERA) that mandated a complete restructuring of the public elementary and 
secondary system in the areas of finance, governance, curriculum, and 
assessment. The school district in Kentucky that served as research site for this 
study has more than 93,000 students in grades K-12 and approximately 150 
school sites. The district differs from other school systems since it has a student 
assignment plan based on managed choice that produces racial disaggregation of 
its schools by providing transportation of students from their home neighborhoods 
to other parts of the district. 
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This third-party investigation contributes to extant research on SFA by 
using a mixed method approach for a large number of school-related variables. 
The study’s primary objective was determining the impact of the program on 
school, student, teacher, and parent variables after three years of implementation. 
The following overarching research questions guided the study: Does SFA impact 
students’ reading achievement? Does SFA result in improved student attendance? 
Does the model result in a decrease in discipline problems? Finally, what affects 
does SFA have on teacher, student, and parent perceptions of school climate, 
educational quality and job satisfaction in their schools? 

Since SFA is primarily a reading program implemented during the school 
hours, the analysis of the impact of the program in reading scores was an essential 
component of the study. Supplementary variables such as attendance, 
suspensions, and school climate were assessed to gain understanding on the 
systemic nature of change that SFA creates when implemented in public schools. 
SFA is a program that is comprehensive in nature (e.g., curriculum, assessment, 
individualized assistance, family support). In this regard, the framework of 
biosocial developmental contextualism (Ramey & Ramey, 1998) provided a 
rationale for assessing the aforementioned accompanying variables. 
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Method 

Participants 

The sample included three treatment elementary schools and three 
matched control schools. The three schools became involved with SEA as part of 
their effort to increase student achievement and had been implementing SFA for 
three years. Previous research methodology served as the basis for developing the 
matching procedure (Borman & Hewes, 2001; Madden, Slavin, Karweit, Dolan, 
& Wasik, 1993; Ross et al, 1997). Schools were matched on demographic 
characteristics of their students from data collected in the baseline year (1999- 
2000). Table 1 illustrates the key characteristics in which the treatment and 
control schools were matched, the aggregated data for both treatment and control 
schools, and the district average. The variables included are poverty, mobility, 
attendance. Exceptional Child Education (ECE), single parent household, and the 
CTBS reading scores. The urban district that served as research site is 
characterized by the high poverty level of its student population. Table 1 showed 
that the treatment and control schools had an even higher percentage of students 
on poverty when compared to the district average of free/reduced lunch 
participants. 

[Insert Table 1 about here] 

Table 2 summarizes the baseline data for the student level samples. As it 
can be observed the treatment and control group were similar in demographic 
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variables. The previous test scores were found dissimilar and a statistical 
adjustment was deemed necessary to control for initial differences. 

[Insert Table 2 about here] 

Instrumentation 

All data, from 1997-1998 (baseline year) to 2000-2001 school years, were 
abstracted from computerized files provided by the school district that served as 
the research site. Data from the state assessment system (i.e., CTBS) and from the 
school district (reading diagnostic test) were utilized. The primary dependent 
variable used in this study was the CTBS Normal Curve Equivalent (NCE) scores 
in reading (Kramer, Conoley, & Murphy, 1992). NCE scores ranges from 1 to 99 
with an average of 50; these scores compare to the students performance to a 
national norm group. The CTBS is a standardized achievement test that was 
group-administered at the end of the school year. The CTBS includes reading, 
language arts, and mathematics sub-tests. The questions are multiple-choice. The 
level 13 tests are given to the elementary school third graders and have 30 items 
in reading. Since SEA is a reading curriculum based on research and effective 
practices (Madden & Slavin, 1999), only the reading indices were abstracted. 
Non-cognitive indicators such as attendance and suspensions were also utilized as 
dependent variables. 

The fundamental independent variable was membership to treatment and 
control groups, using the Stanford Reading Diagnostic Test (SDRT) as the key 
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covariate. The SDRT (Kramer, Conoley, & Murphy, 1992) is standardized, 
diagnostic test that was administered to second grade students during the first few 
weeks of the school year. . 

In an attempt to capture teacher (Appendix A), student (Appendix B), and 
parent (Appendix C) perceptions, data were collected from the district- wide 
comprehensive surveys. The surveys contained different subscales, such as school 
climate and educational quality with Likert-type response scales ranging from 1 = 
Strongly Disagree to 5 = Strongly Agree (see Appendix A, B, and C for scales 
and items). Teachers were also asked about their job satisfaction. 

The results of the reliability analysis conducted on the subscales of the 
teacher, student and parent survey for both the baseline and after three years are 
displayed in Table 3. The reliability analysis yielded alpha coefficients well 
beyond the minimum acceptable level of .60 (Nunnally, 1994) in all the scales of 
the various surveys. 

[Insert Table 3 about here] 

Design and Procedures 

A mixed method design involving qualitative and quantitative research 
proved useful to understand the SFA program in the large urban district that 
served as research site for this study. Few studies have involved the analysis of 
the SFA program, from the perspective of multiple stakeholders (i.e., teachers, 
parents, students). Greene and McClintock (1985) argue that mixed designs can 
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be used for the distinct purpose of complementarity. The concept is to enhance 
and clarify the results from one method with the results from the other method. 
The mixed design was illustrated in this study by the use of a questionnaire about 
school climate and educational quality perceptions of students, teachers, and 
parents combined with more quantitative-oriented measures (e.g., standardized 
test scores, attendance, suspensions). In this study, the analyses were conducted 
separately and integration occurred during interpretation and reporting. 
Furthermore, since results convergence was hypothesized, perception data were 
brought in to support quantitative findings. 

Quantitative Data Analyses. The impact evaluation design employed a 
matched pre-posttest design with control group (Campbell & Stanley, 1963; Cook 
& Campbell, 1979; Kirk, 1990; Rossi, Freeman, & Lipsey, 1999). The data at the 
school level for both treatment and control schools were analyzed using 
descriptive statistics. Due to the inability to randomly assign individuals to 
treatment and control groups, a two-level matching procedure was used to add 
rigor in terms of the internal validity of the study (Campbell & Stanley, 1963; 
Cook & Campbell, 1979). The first level of matching was at the school level and 
it involved checking similarity in terms of poverty (operationalized as 
participation on the free/reduced lunch program), race, single parent households, 
gender, and SDRT scores. These schools, in turn, served as the basis for the 
second level of matching which took place at the student level. Treatment and 
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control students were matched on four demographic variables, namely 
free/reduced lunch, race, single parent households, and gender. Given the 
categorical nature of the variables, the matching procedure was checked using 
chi-squares. No statistically significant differences were found and this confirmed 
that the matching procedure was successful to avoid the need of using statistical 
controls (i.e., covariates) beyond previous test scores. 

The data were first examined to test the statistical assumptions (e.g., 
distributional assumptions of the outcomes, homogeneity of variance, 
examination of outliers) of the desired analysis procedure (Tabachnik & Fidell, 
1996). Since the data were found to be amenable to the General Linear Modeling 
(GLM), the data at the student level was analyzed using a comparison design that 
involved the use of the analysis of covariance (ANCOVA) having the treatment 
condition as the between-subject factor and the pre-test scores as the covariate. 
The ANCOVA (Kirk, 1990) allows for the comparison of group means on a 
dependent variable after the group means have been adjusted on a relevant 
covariate variable. In addition, each mean difference between SFA and control 
students was divided, or standardized, by the pooled posttest standard deviation 
for the outcome (Hedges & Olkin, 1985, p. 79). The resulting standardized 
differences, or effect sizes, provide summaries of the magnitude of each effect 
and are interpretable as the number of standard deviation units separating SFA 
from control students on the outcomes. 
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Qualitative Data Analyses. The comprehensive survey instruments are 
administered yearly using scantron forms at the beginning of the school year. The 
teacher, student and parent surveys are distributed to all schools in October and 
schools are given approximately four weeks to complete and return the surveys to 
the district’s research department. Students are given the parents surveys to take 
home with them and return to school. Schools are asked to create a process that 
maintains confidentiality for staff forms. 

The response rates for the baseline and after three treatment years varied 
among teachers, students, and parents. For the teacher survey was 63.9% for 
controls and 73.7% for the treatment schools for the baseline and treatment school 
years, respectively. The response rate for the student survey was 67.3% for the 
controls and 69.6% for the treatment schools. The response rate for the parent 
survey was 40.8% for the controls and 42.9% for the treatment schools. The 
response rates for the teacher and student surveys were considered “good” 
(Babbie, 1989) while the parent response rate was less than optimal. 

Results 

Research Question 1 ; Does SFA affect the overall student achievement as 
measured on standardized reading test scores? 

Table 4 identifies the CTBS reading scores for both the treatment and 
control schools from 1997-2001. The aggregated treatment school data shows a 
continuous increase in CTBS reading scores from the baseline year to the third 
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year of SFA implementation (4.4 point gain); in contrast, the aggregated control 
schools data shows a gain of only 2.3 points. Two out of three treatment schools 
showed an increase on the CTBS scores during the first year of implementation of 
the model (98-99 to 99-00). Only one treatment school showed gains in the in the 
second year. Two out of three treatment schools showed gains in year 3. 

[Insert Table 4 about here] 

The ANCOVA test was used to analyze the student level sample (N = 

349) and indicated that, after adjustment by the covariate, the effect of the 
program remained significant {F[l, 348] = 4.71, p < .05, ES = .11 ). SFA students 
(M =46.4, SD = 1.2) had higher adjusted mean scores than the control group (M 
= 43.3, SD = 0.9) on CTBS reading scores. 

Research Question 2: Does SFA result in improved school student attendance? 

Table 5 identifies the school level attendance for the treatment and control 
schools. At the school level, the aggregated treatment and control school data 
shows similar trends in attendance. Overall, both treatment and control schools 
made gains in the three years of SFA since the basehne year (1.2 gain and .7 gain, 
respectively). SFA schools doubled the gains of the control schools. 

[Insert Table 5 about here] 

Research Question 3: Does the model result in a decrease of discipline problems? 

From the baseline year to the third year of SFA implementation, the 
aggregated treatment schools more than doubled the decrease in number of out- 
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of-school suspensions when compared to the control schools (7 and 3, 
respectively). Despite the decline, the treatment schools averaged a higher number 
of suspensions. A major increase in suspensions was observed during the first 
year of SFA implementation, particularly in one of the schools. The suspension 
incidences remained higher for the program schools in the second year of SFA. 
The third year of SFA showed a remarkable decrease in number of suspensions at 
the treatment schools. Table 6 displays the discipline data. 

[Insert Table 6 about here] 

Research Question 4: What affects does SFA have on teacher, student, and parent 
perceptions as measured on the comprehensive survey? 

Teachers in the treatment schools gave higher ratings in the areas of 
school climate, educational quality and job satisfaction after three years of SFA 
implementation when compared to the baseline measures (see Table 7). Teachers 
in the SFA schools showed larger improvements in their perceptions of school 
climate, educational quality and job satisfaction than teachers in the control 
schools. Two of the control schools showed decreases in teachers’ perceptions of 
school climate and educational quality. 

[Insert Table 7 about here] 

Students in the treatment schools demonstrated higher ratings of school 
climate and educational quality after SFA implementation than at baseline (see 
Table 8). Again, the results were positive, since the average ratings were above 4 
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on a 5-point scale. While students in the control schools gave higher ratings in 
school climate and educational quality during the 2001-2002 school year than 
students in the treatment schools, students in the SFA schools showed larger 
improvements in their perceptions in both areas from the baseline year. 

[Insert Table 8 about here] 

Parents whose children attended in the treatment schools demonstrated 
higher ratings of school climate and educational quality after SFA implementation 
than at baseline (see Table 9). Parents in the treatment schools gave the same 
ratings in school climate and educational quality during the 2001-2002 school 
year as parents whose children attended the control schools. However, parents in 
the SFA schools showed larger improvements in their perceptions of school 
climate. 

[Insert Table 9 about here] 

In summary, the perceptions of all three key stakeholder groups, teachers, 
students and parents, showed improvements in the areas of school climate and 
educational quality from the baseline year to the after three-year treatment school 
year. In general, the improvements were larger than those of the control schools. 

Discussion 

In an effort to assist schools in making instructional and organizational 
changes, an abundance of school-wide reform models have emerged. With more 
than a decade of research documenting its impact on public education, SFA is 
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considered a tool for urban school reform. The concept of SFA is to restmcture 
elementary schools to ensure that every child learns to read in the early grades. 
The idea is to prevent reading problems from appearing in the first place and to 
intervene speedily and intensively when problems do appear. 

Previous third-party evaluations (Ross, Nunnery, & Smith, 1996; Ross, 
Sanders, & Wright, 1998) of SFA schools in districts across the nation have 
shown that the program increased student reading performance and other school 
related measures. Some SFA research has been criticized in the past. For instance, 
Pogrow (1999) argues that sources of bias are the use of tests that the SFA 
curriculum may be geared to and that control schools do no align with; the fact 
that the vendor selects the test; school sampling bias; student sampling criteria; 
and, the fact that SFA relies primarily on individually administered tests, while 
students are tested in the real world with group administered tests. This study 
addressed those sources of bias by using valid and reliable, group-administered 
tests, namely the CTBS (i.e., achievement test) and SDRT (i.e., diagnostic test). 

This evaluation of the SFA program was conducted at both school and 
student level. The school-level showed that the model has a positive effect on 
CTBS scores and attendance, but it varied across schools. The treatment schools 
that have properly implemented SFA have continuously increased their CTBS 
scores in comparison to the control schools. In terms of attendance, the gains are 
also higher for the treatment schools when compared to the control schools. In 
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terms of suspensions, a notable decrease was experienced in the treatment schools 
when compared to the control schools. At the student level, the results showed a 
reading effect in the SFA students when compared to the control students. The 
connection between SFA and CTBS lies in the basic skills that are tested in CTBS 
and the skills taught in the SFA curriculum. Teachers, students and parents at 
SFA schools showed larger improvements in their perceptions of school climate, 
educational quality, and job satisfaction when compared to those of the control 
schools. 

These findings agree with the recent research findings of other third-party 
meta-evaluators. Herman, Aladjem, McMahon, Masem, Mulligan, O’Malley, 
Quinones, Reeve, and Woodruff (1999) conducted a study that evaluated the 
research base underlying the most widely used school wide programs. Herman 
and colleagues concluded that SFA is one of three programs with strong research 
support of effectiveness. More recently, a meta-analysis conducted by Borman, 
Hewes, Overman, and Brown (2002) found that SFA is one of the three models 
meeting the highest standard of evidence on improving student test scores across 
varying contexts and study designs. 

The framework of biosocial developmental contextualism (Ramey & 
Ramey, 1998) provides an avenue for understanding the program benefits 
(Borman & Hewes, 2001). The framework highlights issues such as (a) 
developmental timing; (b) program intensity; (c) direct provision of learning 
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experiences; (d) program breadth and flexibility; (e) individual differences in 
program benefits; and, (f) environmental maintenance of development. 

The overall positive outcomes might be associated to multiple factors. 

SFA is a highly scripted program supported by on-site facilitators that provide on- 
going professional development. SFA emphasizes attention to the critical first 
grade of elementary school (Wasik & Slavin, 1993): certified teacher tutors (not 
teacher aides) work intensively with early-identified academically at-risk students 
and provide direct learning experiences. The program is comprehensive in nature 
and includes continuous assessment, individualized assistance, and family 
support. SFA produced these types of school outcomes partially because of the 
quality implementation, strong external technical assistance and district-level 
support. Principals of the SFA schools are ensuring the uninterrupted 90-minute 
block and monitoring the implementation at the classroom level. Teachers have 
voted to implement the program to ensure their empowerment. 

Although clearly encouraging, the findings of this mixed method study are 
limited in many important ways. This study was not conducted as a randomized, 
controlled trial, which limits the ability to reach firm causal conclusions about 
intervention effectiveness. Due to the strong matching procedure and statistical 
controls, the study still has strong internal validity and differential group change 
may be attributed more to the interventions than to potentially confounding 
intervening factors (Cook and Campbell, 1979). The perception data associated 
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with parents, due to the low response rate, should not be generalized to other 
settings. 

Regardless of the strong matching procedure, future third-party 
researchers need to address the impact of the program on higher-order skills 
assessed using open-ended questions. This is important in light of the value the 
open-ended responses get in the Commonwealth Accountability Testing System 
used in Kentucky. KERA, one of the nation’s most comprehensive and longest- 
mnning statewide school-reform, places high weight to the criterion-referenced 
test used to specifically assess the educational standards delineated in the 
Kentucky Academic Expectations and the Core Content for Assessment (Pankratz 
& Petrosko, 2000). In addition, given the dynamic school characteristics and 
internal factors affecting model implementation, further analysis using qualitative 
methods might clarify factors related to implementation quality, including 
principal support and buy-in on the part of teachers. 

The impact of SFA is important in light of the accountability and reform 
movement in the schools, especially in a state like Kentucky. The function of an 
accountability system in education is to monitor and evaluate the performance of 
the education system as a whole and the individual school’s achievement 
(Wohlstetter, 1991). Outcome-based education focuses on defining specified 
educational results. Across the nation the clamor is to raise results for all our 
children, regardless of their socio-economic conditions. 
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Appendix A: Teacher Survey 

School Climate 

1. Our school recognizes student’s outstanding contributions or performance. 

2. All students in our school are capable of learning. 

3. Our staff actively supports the school’s Consolidated Plan. 

4. My principal and other building administrators are supportive of my 
efforts to educate students. 

5. My school building is clean and well maintained. 

6. Overall, I am satisfied with the quality of communications within the 
district. 

7. Parents of my students invest time or resources in our school’s program. 

8. Students in my school are disciplined fairly and effectively. 

9. My school administration does a good job of managing this school. 

10. Parents, students, and staff all work together to improve the school. 

1 1 . Resources at my school are distributed fairly. 

12. My school’s staff and faculty work together effectively. 

13. Faculty and staff at my school set high expectations for the achievement 
and social development of each student. 

14. My students are motivated to learn. 

15. Students are proud to attend my school. 

16. Most of our students obey the rules established by the faculty and staff. 

17. Faculty and staff have high expectations for one another 

18. A positive atmosphere for learning exists at my school. 

19. Faculty and staff at my school really care about students as persons. 

20. 1 am proud of my school. 

21.1 feel confident using technology in my classroom. 

22. 1 am able to provide a variety of learning environments to meet the needs 
of different learners. 

23. Our school is effectively implementing a plan to close the achievement 
gap. 

Educational Quality 

1. A strong academic program is offered to students at my school. 

2. Curriculum offerings in our school meet the needs of most students. 

3. Weekly, my students receive feedback regarding their academic 
performance. 

4. Our school has implemented an effective Extended School Services 
Program that meets student needs. 

5. My school’s library/Media Center is well equipped to meet student needs. 

6. School -Based Decision-Making has helped improve my school’s quality 
of education. 

7. The quality of education the district provides continues to improve. 
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8. I recommend to parents district’s schools over private and parochial 
schools. 

9. I would rather send my own children to the district’s schools than to 
private/parochial schools. 

10. Our faculty and staff share a commitment to the excellence of the school. 

1 1. 1 am satisfied that my school’s curriculum ensures mastery-level reading 
skills. 

12. 1 am satisfied that my school’s curriculum ensures mastery-level writing 
skills. 

13. 1 am satisfied that my school’s curriculum ensures mastery-level math 
skills. 

14. 1 believe Early Childhood programs in the district have facilitated 
improved achievement in our primary students. 

15. Our school’s curriculum adequately addresses the need to prepare students 
for using technology. 

16. 1 believe our school has fully integrated the curriculum. 

Job Satisfaction 

1 . I am very satisfied with my current position. 

2. I would likely recommend the district as a good place to work. 

3. Overall, I like the work that I am doing. 

4. I would rate my contribution to the success of my school as excellent. 

5. My job gives me adequate opportunities for personal and professional 
development. 

6. I receive adequate training to perform my job well. 

7. My principal gives me adequate feedback on my job performance. 




30 



Educating Students Placed At Risk 30 



Appendix B: Student Survey 

School Climate 

1 . My principal does a good j ob. 

2. The rules at my school are fair to students. 

3. Most students obey school rules. 

4. Most students show the proper respect for adults in my school. 

5. I follow the school rules. 

6. My parent(s)/guardian(s) encourages me to do my best. 

7. My parent(s)/guardian(s) helps me with my homework. 

8. The students in my school are nice to one another. 

9. The adults in my school treat students fairly. 

10. My school is clean. 

1 1. 1 am proud of my school. 

12. The adults in my school are friendly and helpful. 

13. The adults in my school make sure students follow the rules. 

14. My teacher(s) really care about me. 

Quality of Education 

1. The work I do at school helps me learn. 

2. I think I am learning as much as I should. 

3. I enjoy using the library or Media Center when I do my work. 

4. lam happy attending this school. 

5. The work I do in my classes is interesting to me. 

6. The work we do in school will help me get a good job when I grow up. 

7. I get to use technology (computers, VCRs, etc.) in my classes. 

8. My teacher(s) does a good job. 

9. My teacher(s) gives me help when I need it. 

10. My teacher(s) let my parents/guardians know what I am learning in 
school. 

1 1. 1 know what my teacher(s) expects me to do in class. 

12. My teacher(s) expect me to learn a great deal. 

13. My teacher(s) let me know how I am doing with my classes every week. 

14. The work I do at school makes me a better reader. 

15. The work I do at school makes me a better math student. 

16. The work I do at school makes me a better writer. 
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Appendix C: Parent Survey 

School Climate 

1 . Administrators at my child’s school treat my child fairly. 

2. I believe administrators do a good job of maintaining discipline. 

3. The principal makes time to see me when I have concerns. 

4. My child is given the recognition he/she deserves. 

5. I believe my child is encouraged to do his/her best work. 

6. I regularly help my child with homework. 

7. I am provided adequate information on my child’s progress. 

8. I am able to resolve problems at my child’s school. 

9. I believe my child likes school. 

10. 1 believe a positive learning atmosphere exists at my child’s school. 

11. The school building is clean and comfortable. 

12. Teachers at my child’s school treat my child fairly. 

13. 1 believe teachers do a good job of maintaining discipline. 

14. 1 feel the teachers care about my child as a person. 

15. The teacher(s) makes time to talk with me when I have concerns. 

16. The staff is friendly and helpful when I call or visit the school. 

17. 1 have the opportunity to help make decisions on how the school is 
managed. 

18. Our school’s Site-Based Decision-Making (SBDM) Council makes 
decisions that benefit our school. 

19. The school’s staff and the SBDM demonstrate a commitment to diversity. 
Educational Quality 

1. The work my child does in school is important. 

2. My child is getting a quality education at this school. 

3. My child understands what is required in all classes. 

4. I believe my child will be prepared to go to the next grade level in school. 

5. My child has the opportunity to work on a computer at school. 

6. I would rather my child go to this school than any other public school. 

7. I would rather my child go to this school than a non-district school. 

8. I am proud of my child’s school. 

9. My child will be able to go to college or get a job after graduation from 
the district. 

10. 1 am aware that my child must complete a writing portfolio. 

11. My child is writing more at home and at school than in previous years. 

12. 1 believe my child’s teachers want her/him to be successful. 

13. The teaching methods used by teachers are very effective for my child. 

14. My child is reading more at home than in past years. 

15. My child is doing more mathematics assignments at home than ever 
before. 




32 



Educating Students Placed At Risk 32 



Table 1 

Treatment and Control Schools Matched at Baseline Year tN = b') 



School 


Povertv Mobilitv 


Percent 

Attendance ECE 


Single 

Household 


CTBS 


School A 


86.0 


17.89 


92.8 


8.32 


63.5 


38.1 


Control A 


88.8 


18.93 


94.1 


6.73 


66.0 


38.0 


School B 


81.1 


16.27 


94.3 


6.27 


71.7 


38.3 


Control B 


83.2 


10.98 


94.3 


7.79 


60.2 


40.4 


School C 


75.3 


15.90 


93.3 


7.09 


66.8 


43.0 


Control C 


73.5 


16.93 


94.3 


5.35 


68.8 


36.9 


Avg. Treatment 


80.8 


16.69 


93.5 


7.23 


67.3 


39.8 


Schools 


Avg. Control 


81.8 


15.61 


94.2 


6.62 


65.0 


38.4 


Schools 


District Avg. 


56.0 


10.37 


95.0 


6.62 


49.4 


48.2 



Table 2 

Baseline Data for the Student Level Analytical Sample (N = 3491 



Success for All (n = 217) Control (n = 132) 





M 




M 


SD 


Reading Pretest 


37.00 


16.80 


35.49 


15.64 


Female 


0.55 


0.50 


0.55 


0.50 


Minority 


0.54 


0.50 


0.59 


0.49 


Free/Reduced Lunch 


0.86 


0.35 


0.83 


0.37 


Single Parent Home 


0.73 


0.44 


0.72 


0.45 



ERIC 
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Table 3 



Reliability Coefficients f o r Teacher. Student and Parent Comprehensive Surveys 



Survey Number of 

Respondents 
Baseline Year 3 


Number of 
Items 


Cronbach Alpha 
Baseline Year 3 


Teacher Survey 


School Climate 115 


106 


25 


.93 


.94 


Educational Quality 115 


98 


17 


.94 


.95 


Jobs atisf action 115 


107 


7 


.95 


.89 


Student Survey 


School Climate 667 


522 


14 


.84 


.86 


Educational Quality 667 


547 


16 


.88 


.90 


Parent Survey 


School Climate 867 


565 


19 


.94 


.95 


Educational Quality 867 


565 


15 


.97 


.92 



Table 4 










CTBS Reading NCE Scores for Treatment and Control Schools TN - 61 


^ 


School Baseline Year 


SEA Year 1 


SFA Year 2 


SFA Year 3 


School A 


30.1 


38.1 


34.6 


35.1 


Control A 


38.2 


38.0 


39.4 


39.0 


School B 


40.8 


38.3 


47.0 


51.3 


Control B 


43.1 


40.4 


44.2 


43.4 


School C 


40.1 


43.0 


39.9 


37.9 


Control C 


36.3 


36.9 


39.4 


42.1 


Aggregated 

Treatment 


37.0 


39.8 


40.5 


41.4 


Aggregated 

Control 


39.2 


38.4 


41.0 


41.5 
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Table 5 

School Attendance Percentages for Treatment and Control Schools CN = 6^) 
School Baseline Year SPA Year 1 SPA Year 2 SPA Year 3~ 



School A 


92.8 


93.1 


93.0 


93.9 


Control A 


94.1 


93.6 


94.4 


94.4 


School B 


94.3 


93.3 


95.1 


95.6 


Control B 


94.3 


93.3 


94.9 


95.4 


School C 


93.3 


93.5 


93.8 


94.7 


Control C 


94.8 


94.8 


94.8 


95.4 


Aggregated 

Treatment 


93.5 


93.3 


94.0 


94.7 


Aggregated 

Control 


94.4 


93.9 


94.7 


95.1 


District 


92.7 


93.1 


93.4 


94.1 



Table 6 

School Suspension Incidences for Treatment and Control Schools TN = b') 



School 


Baseline Year SPA Year 1 


SPA Year 2 SPA Year 3 


School A 


27 


58 


27 


15 


Control A 


6 


27 


10 


5 


School B 


6 


1 


3 


4 


Control B 


0 


1 


2 


3 


School C 


16 


8 


22 


7 


Control C 


16 


5 


6 


3 


Aggregated Treatment 


16 


22 


17 


9 


Aggregated Control 


7 


11 


6 


4 


District Avg. 


2 


2 


2 


2 
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Table 7 

Teacher Perceptions for Treatment and Control Schools TN = 6) 



School 



School Climate 





Baseline 


Year 3 


Baseline 


Year 3 


Baseline 


Year 3 




MrSDl 


MfSDl 


M(SD) 


M(SD') 


MfSDl 


M(SD1 


School A 


4.3 (.5) 


4.4 (.5) 


3.6 (1.2) 


4.4 (.6) 


3.9 (1.4) 


4.6 (.5) 


Control A 


4.5 (.7) 


3.5 (.7) 


4.0 (1.4) 


3.5 (.7) 


5.0 (0.0) 


4.5 (.7) 


School B 


4.1 (.7) 


4.3 (.6) 


4.2 (.8) 


4.3 (.7) 


4.5 (.6) 


4.4 (.7) 


Control B 


4.0 (.7) 


3.6 (.9) 


3.9 (.8) 


3.8 (.7) 


4.4 (.5) 


4.4 (1.1) 


School C 


3.8 (.9) 


4.1 (.7) 


3.6 (1.0) 


4.0 (.8) 


4.0 (.8) 


4.6 (.7) 


Control C 


4.0 (.8) 


4.2 (.7) 


3.9 (.9) 


4.0 (.7) 


4.3 (.6) 


4.5 (.5) 


Aggregated 














Treatment 


4.1 (.7) 


4.3 (.6) 


3.9 (1.0) 


4.3 (.7) 


4.1 (1.0) 


4.5 (.6) 


Aggregated 














Control 


4.0 (.8) 


4.0 (.8) 


3.9 (.9) 


3.9 (.7) 


4.4 (.6) 


4.5 (.7) 
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Table 8 

Student Perceptions for Treatment and Control Schools TN = 61 



School School Climate Educational Quality 

Baseline Year 3 Baseline Year 3 

„ M (SD) M rSDl MrSD^ MfSD^ 



School A 
Control A 


3.9 (.8) 
4.1 (.8) 


4.1 (.8) 

4.2 (.5) 


4.3 (.8) 
4.3 (.9) 


4.4 (.8) 

4.5 (.5) 


School B 
Control B 


4.4 (.6) 
4.3 (.8) 


4.3 (.7) 
4.2 (.7) 


4.6 (.6) 
4.5 (.7) 


4.6 (.6) 
4.5 (.5) 


School C 
Control C 


3.8 (.8) 
4.4 (.7) 


4.3 (.7) 
4.3 (.7) 


4.1 (.9) 
4.6 (.6) 


4.4 (.9) 
4.6 (.6) 


Aggregated Treatment 
Aggregated Control 


4.1 (.8) 
4.3 (.7) 


4.2 (.7) 

4.3 (.7) 


4.3 (.8) 
4.5 (.7) 


4.5 (.8) 

4.6 (.6) 
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Table 9 

Parent Perceptions for Treatment and Control Schools TN = 61 



School 


School Climate 


Educational Oualitv 




Baseline 


Year 3 


Baseline Year 3 




MfSDl 


MrSDl 


M fSDl M fSDl 


School A 


4.0 (.8) 


4.4 (.7) 


4.1 (.8) 4.4 (.7) 


Control A 


4.0 (.6) 


4.4 (.7) 


3.9 (.6) 4.4 (.6) 


School B 


3.9 (.7) 


4.5 (.6) 


4.0 (.7) 4.6 (.6) 


Control B 


4.2 (.7) 


4.4 (.6) 


4.2 (.7) 4.3 (.7) 


School C 


4.1 (.7) 


4.4 (.7) 


4.2 (.7) 4.4 (.7) 


Control C 


4.3 (.7) 


4.5 (.8) 


4.2 (.7) 4.5 (.7) 


Aggregated Treatment 


4.0 (.8) 


4.4 (.7) 


4.1 (.8) 4.4 (.7) 


Aggregated Control 


4.2 (.7) 


4.4 (.7) 


4.1 (.7) 4.4 (.7) 
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